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Recent tests performed on the D-Wave Two quantum annealer have revealed no clear evidence of speedup 
over conventional silicon-based technologies. Here, we present results from classical parallel-tempering Monte 
Carlo simulations combined with isoenergetic cluster moves of the archetypal benchmark problem—an Ising 
spin glass—on the native chip topology. Using realistic uncorrelated noise models for the D-Wave Two quantum 
annealer, we study the best-case resilience, i.e., the probability that the ground-state configuration is not affected 
by random fields and random-bond fluctuations found on the chip. We thus compute classical upper-bound 
success probabilities for different types of disorder used in the benchmarks and predict that an increase in the 
number of qubits will require either error correction schemes or a drastic reduction of the intrinsic noise found 
in these devices. We restrict this study to the exact ground state, however, the approach can be trivially extended 
to the inclusion of excited states if the success metric is relaxed. We outline strategies to develop robust, as 
well as hard benchmarks for quantum annealing devices, as well as any other (black box) computing paradigm 
affected by noise. 


I. INTRODUCTION 

Although a useful universal quantum computer QH is far 
from reality at the moment, the advent of quantum anneal¬ 
ing (QA) machines based on quantum adiabatic optimization 
techniques [H-QaI] has sparked a small computing revolution 
in recent years. Being a novel hardware based on nonsilicon 
chips used to perform computations exploiting the potential 
advantages of quantum fluctuations fl5ll . quantum annealing 
machines might affect the way a multitude of hard optimiza¬ 
tion problems are solved today. 

The first somewhat useful programmable commercial de¬ 
vices that attempt to exploit this unique power are the D-Wave 
One and Two quantum annealers [16], that are designed to 
solve quadratic unconstrained binary optimization (QUBO) 
problems d, such as finding the ground state of a disor¬ 
dered Ising spin-glass Hamiltonian, a well-known NP-hard 
problem in this general formulation [|T§]. Because many prob¬ 
lems across disciplines can be mapped onto QUBOs, multi¬ 
ple studies of the D-Wave quantum annealer’s performance, 
compared to some classical optimization approaches, such as 
simulated annealing (SA) JTp]]. have been performed l l20H3lll . 
Tests lf2ll I 22 I [ 24 I l27tl by different research teams suggest that 
the D-Wave quantum annealer does benefit from quantum ef¬ 
fects. However, it is unclear if this quantum advantage is in¬ 
volved in the optimization of cost functions. Furthermore, to 
date these studies reveal no clear evidence of limited quantum 
speedup [[26] over classical optimization algorithms on tradi¬ 
tional computers. 

Recent work by Katzgraber et al. l l32tl suggests that current 
benchmarking approaches using spin glasses with uniformly- 
distributed disorder on the Chimera graph [33l, such as bi- 
modal or range-/,:, might not be the best benchmark problems 
in the quest for quantum speedup. In particular. Ref. a 


proposes an innovative approach based on insights from the 
study of spin glasses to design hard benchmark problems 
within the constraints of the D-Wave device. To overcome 
the limitations posed by the D-Wave architecture. Ref. a 
proposes to use instances with a unique ground state, as well 
as many metastable states. In this work we study the inter¬ 
play between the generation of hard benchmark instances with 
the design of problems suitable for the D-Wave device that 
are robust to noise. Ideally, thus, a two-tier (unfortunately 
computationally-expensive) data mining approach is needed 
to produce ideal test instances for any quantum annealing de¬ 
vice: First, random benchmark instances are mined for their 
desired properties (e.g., unique ground state) that make them 
hard problems to solve. Second, these instances are tested for 
their robustness to the intrinsic noise present in any hardware 
device. 

The fact that different numerical studies H 03 0 0 
demonstrated that QA might outperform SA in certain 
problems—especially those with rough energy landscapes— 
has motivated the authors of Refs. [[341] and I 36TI to design tun¬ 
able hard benchmarking problems. Reference [13411 goes a step 
further, by being able to carefully tune the barrier thickness 
between dominant features in the energy landscape, thus pu¬ 
tatively allowing for the detection of any quantum advantage 
that a quantum annealing device might pose over traditional 
optimization approaches. Despite these efforts, noise due to 
thermal excitations and control errors on qubits and couplers 
have a detrimental effect on the performance of the D-Wave 
quantum annealer 01063 that likely is masking any po¬ 
tential limited quantum speedup 63 - A simple explanation 
for these problems is given by the fragility of spin glasses 
to small perturbations, also known as chaotic effects 141 145 0l] 
to either couplers (bond chaos), qubits via longitudinal fields 
(field chaos), or both couplers and qubits (temperature chaos). 
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Here, small fluctuations can produce large changes in the free 
energy of the system, thus perturbing the original problem 
Hamiltonian to be solved. 

Although quantum error correction |[23l [391:401 can, in prin¬ 
ciple, mitigate these errors, it does so at a cost of needing 
multiple physical qubits to encode one logical qubit, thus re¬ 
ducing the effective system size of problems to be studied. 
This also means that “error-corrected” benchmark instances, 
while more robust to noise, will likely be too small to be in the 
scaling regime of interest for currently available system sizes. 
As such, designing hard benchmark instances that are robust 
to noise and require no overhead in the embedding to keep the 
problem size at a maximum are of utmost importance to detect 
quantum speedup. In this work we classically study resilience, 
i.e., the probability that the ground-state configuration is not 
affected by random fields and random-bond fluctuations found 
on the chip for different benchmark instance classes, by us¬ 
ing realistic uncorrelated noise models for the D-Wave Two 
quantum annealer. Furthermore, we present strategies on how 
to develop hard benchmark instances that, at the same time, 
are robust to noise. Note that our methodology is generic, 
i.e., it can be applied to any architecture or noisy black-box 
optimization device. Furthermore, the study can be trivially 
extended to include low-lying excited states if the gold stan¬ 
dard of finding the exact ground state is relaxed to include a 
subset of low-lying excited states. 

The paper is structured as follows. In Sec. [[Owe introduce 
the different benchmark instance classes studied, as well as 
the noise model. Furthermore, we describe the heuristic used 
to find the ground-state configurations. Our numerical results 
on the D-Wave chimera topology are presented in Sec. HIH 
followed by concluding remarks. 

II. MODEL, OBSERVABLES, ALGORITHM 

Our calculations are for the currently-available D-Wave 
Two device EU. However, the ideas can be generalized to 
any topology. 

A. Model 

The native benchmark for the D-Wave Two quantum an¬ 
nealer is an Ising spin glass iQ [52l [53:1 defined on the Chimera 
topology of the system 133]. The Hamiltonian of the problem 
to be optimized is given by 

H = - ^2 JijSiSj-^Sihi. ( 1 ) 

{ i,j}€V i£V 

where s, £ {±1} signify Ising spins on the vertices V of the 
Chimera lattice. Figure [j] shows a 512 qubit Chimera lattice 
with 8 x 8 K 4 4 cells. In addition, each spin Si is coupled 
to a local random field hi. The sum is over all edges £ con¬ 
necting vertices {i,j} £ V.. The interactions J l:l between the 
spins are drawn from carefully chosen, discrete disorder dis¬ 
tributions within the hardware constraints of the D-Wave Two 
architecture. 



FIG. 1: Adjacency matrix of the D-Wave Two chip with 8 x 8 K 4.4 
cells and 512 qubits (circles) connected by couplers (lines). 

To emulate the effects of thermal noise in the device, we 
perturb the discrete values of the couplers J l:j by a random 
amount AJ, ;j drawn from a Gaussian distribution with zero 
mean and standard deviation A J. For simplicity, we assume 
the noise is quenched and uncorrelated. This “white noise” 
represents a realistic (classical) noise model for coupler fluc¬ 
tuations that is typically used to study the effects of noise in 
electronic devices, as well as telecommunications. Although 
the qubit noise in the D-Wave Two device is closer to 1// 
noise with a “pink” power spectrum, for simplicity we couple 
the individual qubits to uncorrelated quenched random fields 
drawn from a Gaussian distribution with zero mean and stan¬ 
dard deviation h. We do not expect this simplification to qual¬ 
itatively change our results. 

B. Instance classes & observables 

Carefully-chosen interactions between the spins determine 
the hardness and robustness of instance classes f34ll . To de¬ 
velop hard instances, multiple requirements have to be ful¬ 
filled. First, it is of paramount importance to ensure that the 
instances have a unique ground-state configuration that min¬ 
imizes the cost function in Eq. ©• Furthermore, it is desir¬ 
able to have dominant metastable states such that the system 
is easily trapped - a process that can be accomplished by a 
post-processing selection and mining of the data based on in¬ 
sights from the study of the dynamics of spin glasses using 
classical simulation techniques |[34^ [540. Ultimately, an ideal 
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benchmark instance is robust to noise, has a unique ground 
state and, ideally, many metastable states. 

To gauge the fraction of unique ground-state configurations 
for a particular instance class, we define a quantity we call 
yield (JO, i.e., 

y = N unique/ N total ■ (2) 

In Eq. Q JVtotai is the total number of randomly-generated 
instances for that particular instance class and iV,„ liq , ]e is the 
number of instances featuring a unique ground state (no de¬ 
generacy). 

One simple approach pioneered in Ref. lf34j to design in¬ 
stance classes with high yield, is to ensure that as few qubits 
Si as possible have zero local fields Ti = Jij s j ± hi. 

If for a given qubit Ti = 0, then the qubit’s value does not 
change the energy of the system. Therefore, if a system with 
N qubits has k free qubits with zero local field, the degen¬ 
eracy of the ground state is increased by a factor 2 k . We 
have exhaustively computed the probability that a particular 
combination of two, three, or four integer values [55] in the 
range {± 1 ,..., ±i max } (with i max = 28) JH| for the cou¬ 
plers Tj on the Chimera topology yields the smallest frac¬ 
tion of qubits with zero local fields. Furthermore, we have 
attempted to “spread out” the integers as much as possible in 
the range [— 1 , 1 ] after a normalization of the coupler values 
with * max . In addition to the previously-studied cases of bi- 
modal disorder, i.e., 

Ui G {±1}, 

as well as uniform range -k disorder with k = 1 i2ll.l26ll 
U 4 G {±1, ±2, ±3, ±4}, 

we also study Sidon-type instances J34,i57i], namely 
U 5 , 6,7 G {±5, ± 6 , ±7}, 

which are similar to uniform range-7 instances, however only 
the three largest integers that form a Sidon set are kept. Fi¬ 
nally, we study a larger Sidon set 

S 28 G {± 8 , ±13, ±19, ±28}. 

The 115 , 6,7 and S 2 8 Sidon instance classes reduce the proba¬ 
bility of zero local fields drastically by design, and thus max¬ 
imize the yield of unique ground states. In fact, while Ui has 
an average probability of 23% to have zero local fields, this 
number is reduced to 6 % in the U 4 class. 115 , 6,7 has only 
4.5% zero local fields and S 2 8 has 1.5%. 

To increase the resilience to noise for a given instance, one 
has to maximize the change in energy when flipping a spin, 
i.e., the minimum classical energy gap. Ideally, this change 
in energy should be considerably larger than the typical noise 
fluctuations to prevent qubit errors. For Ising spins, this en¬ 
ergy gap is given by A E = 2/j max , where * max is the largest 
integer in the unnormalized bond distribution. For example, 
AE(Ui) = 2 , whereas AE(XJ 4 ) = 1/2, AE(U 5 , 6i7 ) = 2/7, 
and AE(S 2 8 ) = 1/14 ~ 0.07. For the current D-Wave Two 
machine with 512 qubits, coupler fluctuations are typically 


~ 0.035 if the bonds are normalized to unity (“autoscaling 
mode”). This means that in this case the S 2 8 instance class 
pushes the limits of the machine because AE(S 2 s) ~ 2 A J. 

To quantify the robustness of ground-state configurations to 
noise, we define the resilience R of an instance to be 

R = hJ same/ N trials (3) 

where N same is the number of trials with different ran¬ 
dom noise perturbations (either fields or bonds) that do not 
change the original ground-state configurations. We perform 
Aerials = 10 trials (or gauges) to compute R. The resilience 
of an instance class is the resilience for each instance R av¬ 
eraged over the bond disorder, i.e., 1Z = [f?] av , where [• • • ] av 
represents an average over multiple random bond configura¬ 
tions. A preference should be given to whole instance classes 
with high resilience. However, individual instances that are 
unaffected by the perturbations are also robust instances and 
can be used for benchmarking purposes. Conversely, to study 
the effects of noise in quantum annealing machines and how 
to reduce these, instances with a small resilience can also be 
mined ll29ll . 

Finally, we emphasize that a “relaxed” resilience Rk can 
also be defined, where 

Rk = A r same (E < E}- ) /N trials • (4) 

Here N same (E < Ef.) is the number of times a state with 
an energy E less or equal than the energy of the fc-th excited 
state is found. This is of importance when the analog machine 
suffers from high noise levels and where the determination of 
the exact ground state is difficult or even impossible. For dis¬ 
crete disorder distributions—as commonly used on quantum 
annealing machines with finite precision—the energy levels 
are separated by well-defined values, i.e., computing the re¬ 
laxed resilience of an instance class, 7 Zk = [-Rfc] av , is well 
defined. 


C. Algorithm details 

In order to measure the yield and resilience of a particu¬ 
lar instance class, ground states of instances from all instance 
classes have to be found. We apply a heuristic method that 
uses the parallel tempering Monte Carlo algorithm |l58j] com¬ 
bined with isoenergetic cluster moves [59j| to speed up the 
thermalization. Simulation parameters are listed in Table [I] 
and thermalization has been determined by a logarithmic bin¬ 
ning of the data. Once the last three bins agree within error 
bars, we deem the system to be in thermal equilibrium. The 
detailed algorithm to detect ground states was first introduced 
in Ref. |60j]. However, to increase the accuracy of our heuris¬ 
tic, here four instead of two copies of the system with the 
same disorder are simulated with independent Markov chains. 
We perform 7V SW updates For N sw /8 updates we keep 

track of the lowest energy E of each Markov chain at the low¬ 
est temperature simulated. If E ^ = E^ = E^ = E^ 4 \ 
it is very likely the ground state energy Eq has been found. 
For the remaining number of updates we keep statistics of the 
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TABLE I: Simulation parameters: For each instance class and sys¬ 
tem size N, we compute N a a instances. N sv , = 2 6 is the total num¬ 
ber of Monte Carlo sweeps for each of the ANt replicas for a single 
instance, T m i n [TAax] is the lowest [highest] temperature simulated, 
and Nt is the number of temperatures used in the parallel temper¬ 
ing method. For the lowest Mem temperatures isoenergetic cluster 
moves are applied. 


Class 

N 

M sa 

b 

T m i n 

Tm ax 

N t 

A icm 

Ui 

512 

900 

19 

0.150 

3.050 

30 

13 

u 4 

512 

900 

19 

0.150 

3.000 

30 

14 

Us, 6,7 

128 

900 

19 

0.150 

3.000 

30 

14 

Us, 6,7 

288 

900 

19 

0.150 

3.000 

30 

14 

US,6,7 

512 

900 

19 

0.150 

3.000 

30 

14 

US,6,7 

800 

900 

19 

0.150 

3.000 

30 

14 

US,6,7 

1152 

900 

19 

0.150 

3.000 

30 

14 

S 28 

512 

900 

19 

0.150 

3.000 

30 

14 


configurations that minimize the Hamiltonian and thus esti¬ 
mate the degeneracy distribution of the ground state. How¬ 
ever, there is no guarantee that any solution obtained by this 
heuristic method is the true optimum, or that we have found 
all configurations that minimize the Hamiltonian. Fortunately, 
for the Sidon-type instance classes the degeneracy is small by 
construction. Therefore, it is likely that we found all ground- 
state configurations. Once the ground-state configurations of 
all instances have been found, the average yields for different 
instance classes can be computed. 

In addition to the effects of the minimum energy gap A E on 
the resilience for each instance class, we also consider the ef¬ 
fects of the number of first excited states on the resilience. To 
estimate the number of first excited states, for the remaining 
(7/8) N SVJ sampling updates we also keep track of all config¬ 
urations that have an energy E\ = Eq + A E. 


HI. RESULTS 

We focus only on the resilience of the exact ground state in 
this study for the sake of brevity and to illustrate the developed 
methodology. Our approach is easily extended to include low 
excited states. Note that if the resilience R of an instance is 
large, we also expect the relaxed instance resilience Rk to be 
large for small enough k. 


A. Yield of non-degenerate ground states 

For the current D-Wave Two architecture with 512 qubits, 
the yield of unique ground states is strongly dependent on the 
instance class (disorder between spins) used. When the disor¬ 
der is drawn from a bimodal distribution (Ui) the yield in all 
our experiments was exactly 0%. Surprisingly, uniform range- 
4 instances (U 4 ) also have y = 0%. However, by increas¬ 
ing the range of the integers and selecting them from a Sidon 
set while removing the lowest values gives y = 4.5(4)% 
for the 115 , 6,7 class. Although a small fraction, it is clearly 
nonzero. Finally, for the large Sidon set S 28 we obtain a frac¬ 
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FIG. 2: (Color online) Resilience ( TZ ) of different instance classes 
(see text) for a N = 512 qubit system on the Chimera graph as a 
function of Gaussian random field strength ( h ). Instance classes are 
less resilient to noise with increasing field strength and decreasing 
classical energy gap. The shaded line represents the current field 
noise strength of approximately 5% in the D-Wave Two system. 

tion y = 20 . 0 ( 6 )% of unique ground states |62j], i.e., optimal 
for large-scale benchmarking. 


B. Resilience to noise 

Figure [2] shows the resilience to random-field noise for dif¬ 
ferent instance classes. As the typical field strength h in¬ 
creases, the resilience TZ for all instance classes decreases. 
This is to be expected, because the energy spread due to the 
splitting of degenerate excited states via the random fields re¬ 
sults in more energy levels crossing. Furthermore, for a fixed 
field strength, instance classes with small energy gaps A E 
tend to have lower resilience. This is to be expected: it is 
easier for split states to have a lower energy than the original 
ground state when the gap is small. Note that while instance 
classes U 5 , 6,7 and U 4 have a similar resilience, the yield of 
unique ground states is considerably higher for Us, 6 , 7 , he., 
a careful design of the spin-spin interactions is key when at¬ 
tempting to benchmark a quantum annealing device. 

Figure [3] shows the resilience of different instance classes 
as a function of different typical coupler perturbations A J. 
Again, for all instance classes studied, the resilience decreases 
as fluctuations increase. In addition, instance classes with 
small energy gaps have a lower resilience. It is important to 
note that bond noise has a stronger impact on the resilience 
than field noise. Considering each qubit has typically ~ 6 
neighbors in the Chimera lattice, the impact of bond noise is 
amplified by multiple connections of qubits. Therefore, re¬ 
ducing the fluctuations of the couplers is more important than 
dealing with the intrinsic flux noise of each qubit. 

Unfortunately, for the D-Wave architecture, to find an in¬ 
stance class that is both hard and robust to noise, compro¬ 
mise has to be made. The Ui instance class has the highest 
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FIG. 3: (Color online) Resilience (72) of different instance classes 
(see text) for a N = 512 qubit system on the Chimera graph as a 
function Gaussian random bond fluctuation strength (AJ). Instance 
classes are less resilient to noise with increasing bond fluctuation 
strength and a decreasing classical energy gap. The shaded line rep¬ 
resents the current bond noise strength in the D-Wave Two system, 
i.e., ~ 3.5%. Note that bond noise has a stronger effect than field 
noise (FigO on the device. 


resilience to noise, however, the huge ground-state degener¬ 
acy makes it easier for classical algorithms such as SA to find 
minimum-energy configurations 132. 13411 . On the flip side, 
the Sidon instance class is known to be hard El and pro¬ 
duces many unique ground states, but its resilience is compa¬ 
rably low due to the small energy gap. A compromising nat¬ 
ural choice would therefore be to either use the U 4 or 1 ^ 6,7 
instance classes. However, while the resilience for both U 4 
and 115 , 6,7 are comparable, the yield of unique ground states 
needed to construct hard benchmark problems is much higher 
for 115 , 6 , 7 . We thus conclude that for the current Chimera 
topology, the 115 , 6,7 instance class is the optimal compromise 
to design hard benchmark problems within the D-Wave Two 
architecture constraints. For the remainder of this paper we 
thus focus on this particular instance class. 

Figure [4] shows the resilience of the 115 . 6,7 instance class 
for different system sizes N of the Chimera lattice as a func¬ 
tion of the random-bond fluctuation strength A J. Clearly, for 
increasing system size the resilience 1Z decreases (larger sys¬ 
tem sizes typically have a higher degeneracy, therefore level 
crossings are more common than with smaller systems). This 
means that to scale up the system size of the D-Wave Two— 
or any other quantum annealing device—in the future, a much 
more precise control over the device’s noise and/or the imple¬ 
mentation of error correction schemes I 23 I [39 . 40ll are imper¬ 
ative. 

We conclude this section by quoting results specifically cal¬ 
culated for the current D-Wave Two (512 theoretical qubits) 
and the next-generation D-Wave 2X (1152 theoretical qubits) 
machines when both errors in the couplers and qubits are ap¬ 
plied, using the real values provided by D-Wave, Inc. [63 ]. For 
the D-Wave Two machine with 512 qubits, A J = 3.5% and 


FIG. 4: (Color online) Resilience 72 of the Us, 6,7 instance class as 
a function of the bond fluctuation strength (A J) for different system 
sizes N on the Chimera topology. The resilience clearly decreases 
for increasing noise and system size. The shaded vertical line rep¬ 
resents the current bond-noise strength in the D-Wave Two system, 
approximately 3.5%. 

Ah = 5%. Applying both coupler and qubit perturbations 
yields an average resilience of 7 Z = 0.22(2). For the next- 
generation D-Wave 2X device noise levels have been reduced, 
i.e., A J = 2.5% and Ah = 3%. This results in 72 = 0.21(3). 
We point out two interesting facts: First, it seems that the re¬ 
silience for both coupler and qubit noise is approximately the 
product of the resilience of only noise being considered on 
the couplers with the resilience of only noise being consid¬ 
ered on the qubits. Thus, as a rule of thumb and to obtain an 
approximate estimate for the combined effects, the individual 
numbers can be multiplied. Second, despite the lower noise 
level of the next-generation device, the resilience remains ap¬ 
proximately unchanged within error bars. It seems that the 
increased number of qubits cancels out the additional preci¬ 
sion. 


C. Effects of the number of first excited states 

Figure [5] shows the resilience 72 of the U 5 , 6,7 instance class 
as a function of the degeneracy of the first excited state on the 
Chimera topology with N = 512 spins. The higher the degen¬ 
eracy of the first excited state, the lower the resilience. This 
can be explained by the increased probability of level cross¬ 
ing. We also color coded each dot in the figure: The heat map 
represents the number of instances that had a given degener¬ 
acy Ni of the first excited state out of the 900 simulated. In 
this case, the bulk of the instances have between 4 and 8 de¬ 
generate first excited states. This results in a reduction of the 
resilience, compared to instances that contain only one or two 
first excited states. 

While instances with only one or two first excited states are 
extremely rare, the effort needed to find these might outweigh 
the approximately 30% in the resilience reduction by allowing 
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FIG. 5: (Color online) Resilience 1Z as a function of the number of 
first excited states N\ for N = 512 spins on the Chimera lattice. The 
data are for the 115 , 6,7 instance class. The color bar shows approx¬ 
imately how often a given number of first excited states occurs for 
the 900 instances studied. In this case, between four and eight first 
excited states are most common. 


states with three to four first excited states. We thus recom¬ 
mend to fix the number of first excited states to be less than or 
equal to four in this case. 

We have also computed the Hamming distance between the 
ground state and all first excited states for a given instance. 
Our results suggest that when the average Hamming distance 
is small, the resilience to noise is higher. A simple explanation 
is that both ground-state and excited configurations are quite 
similar and therefore the noise affects them comparably, i.e., 
both the ground state and the first excited states are shifted ap¬ 
proximately by the same amount when the Hamming distance 
is small. 


IV. CONCLUSIONS 

In order to develop both hard and robust benchmark in¬ 
stances, we have tested different instance classes by comput¬ 
ing their yield (fraction of instances with a unique ground- 
state configuration) and resilience to noise fluctuations. Ide¬ 
ally, hard instances (high yield) with a high resilience are op¬ 
timal for benchmarking purposes. Both yield and resilience 
can be tuned by a careful design of the instance classes— 
within the hardware restrictions of the machine—followed by 
a mining of the data. Although the numerical effort to do such 
“designer instances” is nonnegligible, we think this is a key 
ingredient in designing good benchmarks for quantum anneal¬ 
ing devices, as well as any other computing architectures. It 
seems that both resilience and yield for the Chimera topology 
are slightly anticorrelated. A good compromise is thus the 
U 5 , 6,7 instance class where Jjj £ {±5, ±6, ±7} that has a 
good resilience to both field and coupler noise, as well as a 
nonzero yield of unique ground states, with a small number of 
first excited states. 

We emphasize that our results for the resilience represent a 
best-case scenario for any quantum annealing machine. Any 


other source of error can only decrease the success probabil¬ 
ities further. However, it could be that the introduction of 
carefully-crafted correlations between bond and field noise 
might reduce the errors and increase the resilience. Bond 
noise is the most limiting issue for the current D-Wave Two 
quantum annealer and is highly dependent on the connectivity 
of the graph. While it is desirable to have a high connectiv¬ 
ity to be able to embed interesting problems on any putative 
architecture, one has to also keep in mind that noise levels 
should be far lower than in the current D-Wave machine. 

This classical study of both resilience and yield plays an 
important role in the design of future adjacency matrices for 
quantum annealing machines, as well as the study of strategies 
to reduce noise in quantum annealers. Our results and meth¬ 
ods can easily be generalized to other systems and thus should 
be of general interest when designing hard instance problems 
that attempt to circumvent the limitations of current hardware. 
Furthermore, calibration of future generations of the D-Wave 
device should be improved to allow for the encoding of more 
complex Sidon sets and thus the design of harder benchmark 
problems. Similarly, although the main goal of this work is 
to produce problems that are robust to noise, the methodol¬ 
ogy presented can be used to design tailored instances that are 
particularly sensitive to noise. This could play an important 
role when designing approaches to better calibrate devices, as 
done in Ref. ||64l l. Finally, we emphasize that if either noise is 
large or the instances produced are too difficult to minimize, 
a relaxed resilience that includes low-lying excited states can 
be defined. 
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